[Bugfix][DeepSeek-V4.1] Honor continue_final_message: render the last assistant turn without EOS - #62887

Overview

The DeepSeek-V4.1 tokenizer (--tokenizer-mode deepseek_v41) ignores continue_final_message. A request that asks the model to continue an assistant prefix ("prefill") is rendered with a closed assistant turn: the prefix ends with <｜end▁of▁sentence｜>, so the model is asked to continue after EOS. This makes the last assistant message wo_eos when continue_final_message is set, which is what DeepSeek's reference encoder does.

Related: #43876 fixes the same gap for the V4 tokenizer (vllm/tokenizers/deepseek_v4.py); this PR is the V4.1 tokenizer (vllm/tokenizers/deepseek_v41.py), which #43876 does not touch. #57882 covered this among other generation controls and was closed unmerged (2026-10-10).

Claims

• continue_final_message=True with a final assistant message renders that message without EOS, byte-identical to the reference encoding of deepseek-ai/DeepSeek-V4.1-Flash-1017 (encoding/encoding.py @ 1bae0e10).
• A final user message, and requests without continue_final_message, render exactly as before.

Validation

• Byte-diff against the reference encoder (CPU). We render a 40-case corpus (tool loops, parallel tool results, reasoning history kept and dropped, mid-conversation system messages, all reasoning efforts, prefill on/off) with vLLM's V4.1 tokenizer, called as the OpenAI frontend calls it, and with the reference encoding.py from the 1017 checkpoint.
  • Before: 4 cases differ.
    • prefill_off and prefill_on end the trailing assistant message with <｜end▁of▁sentence｜>, where the reference leaves it open (wo_eos).
    • Two tool-definition cases differ for an unrelated reason: FunctionDefinition.model_dump() renders "description": null and normalizes key order.
    • This was measured with the V4.1 tokenizer at upstream e68efc685, plus [Bugfix][Frontend] Match DeepSeek-V4.1 encoder to the 1017 reference encoding #62199, which has merged since.
  • After: only the 2 unrelated tool cases differ. Both prefill cases are byte-identical to the reference.
• On hardware, DeepSeek-V4.1-Flash-1017 on 8×H200 (2 engines × TP4, DSpark k=5, kv-cache-dtype fp8_ds_mla), 2026-10-21. A prefill conformance request ("The capital of France is" as the assistant prefix, continue_final_message=true, add_generation_prompt=false) continues the message ("Paris.") on all 4 engines. (We did not run the unpatched tokenizer on DeepSeek hardware; the byte-diff above is the before/after evidence.)
• Unit test: tests/tokenizers_/test_deepseek_v41.py adds a case for chat mode and thinking mode with a final assistant message, plus one showing that a final user message is unchanged.
• ruff check and ruff format --check pass on both files.

Details

Root cause. get_deepseek_v41_tokenizer() builds the conversation for the V4.1 encoder from apply_chat_template(**kwargs), but never reads continue_final_message. The encoder already supports leaving an assistant message open (wo_eos), and the reference sets it for the message being continued.

Fix. Set conversation[-1]["wo_eos"] = True when continue_final_message is true and the last message is from the assistant. That's 4 lines, in the same place the tokenizer already adjusts the conversation (tools on the system message). No frontend or encoder change.

Trade-offs. None known. The flag is only read when the client sets continue_final_message, and the OpenAI frontend already refuses it combined with add_generation_prompt=true (vllm/entrypoints/openai/chat_completion/protocol.py:1041).

How we found it: a byte-level diff of the vLLM V4.1 renderer against the checkpoint's reference encoder, which we run before qualifying a build for production serving.

Contact: rey@cambrialabs.ai